A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations

Matching natural language sentences is central for many applications such as information retrieval and question answering. Existing deep models rely on a single sentence representation or multiple granularity representations for matching. However, such methods cannot well capture the contextualized local information in the matching process. To tackle this problem, we present a new deep architecture to match two sentences with multiple positional sentence representations. Specifically, each positional sentence representation is a sentence representation at this position, generated by a bidirectional long short term memory (Bi-LSTM). The matching score is finally produced by aggregating interactions between these different positional sentence representations, through $k$-Max pooling and a multi-layer perceptron. Our model has several advantages: (1) By using Bi-LSTM, rich context of the whole sentence is leveraged to capture the contextualized local information in each positional sentence representation; (2) By matching with multiple positional sentence representations, it is flexible to aggregate different important contextualized local information in a sentence to support the matching; (3) Experiments on different tasks such as question answering and sentence completion demonstrate the superiority of our model.Comment: Accepted by AAAI-201

A Quantum Many-body Wave Function Inspired Language Modeling Approach

The recently proposed quantum language model (QLM) aimed at a principled approach to modeling term dependency by applying the quantum probability theory. The latest development for a more effective QLM has adopted word embeddings as a kind of global dependency information and integrated the quantum-inspired idea in a neural network architecture. While these quantum-inspired LMs are theoretically more general and also practically effective, they have two major limitations. First, they have not taken into account the interaction among words with multiple meanings, which is common and important in understanding natural language text. Second, the integration of the quantum-inspired LM with the neural network was mainly for effective training of parameters, yet lacking a theoretical foundation accounting for such integration. To address these two issues, in this paper, we propose a Quantum Many-body Wave Function (QMWF) inspired language modeling approach. The QMWF inspired LM can adopt the tensor product to model the aforesaid interaction among words. It also enables us to reveal the inherent necessity of using Convolutional Neural Network (CNN) in QMWF language modeling. Furthermore, our approach delivers a simple algorithm to represent and match text/sentence pairs. Systematic evaluation shows the effectiveness of the proposed QMWF-LM algorithm, in comparison with the state of the art quantum-inspired LMs and a couple of CNN-based methods, on three typical Question Answering (QA) datasets.Comment: 10 pages,4 figures,CIK

Learning Contextualized Document Representations for Healthcare Answer Retrieval

We present Contextual Discourse Vectors (CDV), a distributed document representation for efficient answer retrieval from long healthcare documents. Our approach is based on structured query tuples of entities and aspects from free text and medical taxonomies. Our model leverages a dual encoder architecture with hierarchical LSTM layers and multi-task training to encode the position of clinical entities and aspects alongside the document discourse. We use our continuous representations to resolve queries with short latency using approximate nearest neighbor search on sentence level. We apply the CDV model for retrieving coherent answer passages from nine English public health resources from the Web, addressing both patients and medical professionals. Because there is no end-to-end training data available for all application scenarios, we train our model with self-supervised data from Wikipedia. We show that our generalized model significantly outperforms several state-of-the-art baselines for healthcare passage ranking and is able to adapt to heterogeneous domains without additional fine-tuning.Comment: The Web Conference 2020 (WWW '20

Text Matching as Image Recognition

Matching two texts is a fundamental problem in many natural language processing tasks. An effective way is to extract meaningful matching patterns from words, phrases, and sentences to produce the matching score. Inspired by the success of convolutional neural network in image recognition, where neurons can capture many complicated patterns based on the extracted elementary visual patterns such as oriented edges and corners, we propose to model text matching as the problem of image recognition. Firstly, a matching matrix whose entries represent the similarities between words is constructed and viewed as an image. Then a convolutional neural network is utilized to capture rich matching patterns in a layer-by-layer way. We show that by resembling the compositional hierarchies of patterns in image recognition, our model can successfully identify salient signals such as n-gram and n-term matchings. Experimental results demonstrate its superiority against the baselines

aMV-LSTM: an attention-based model with multiple positional text matching

National audienceDeep models are getting a wide interest in recent NLP and IR state-of-the-art. Among the proposed models, position-based models and attention-based models take into account the word position in the text, in the former, and the importance of a word among other words in the latter. The positional information are some of the important features that help text representation learning. However, the importance of a given word among others in a given text, which is an important aspect in text matching, is not considered in positional features. In this paper, we propose a model that combines position-based representation learning approach with the attention-based weighting process. The latter learns an importance coefficient for each word of the input text. We propose an extension of a position-based model MV-LSTM with an attention layer, allowing a parameterizable architecture. We believe that when the model is aware of both word position and importance, the learned representations will get more relevant features for the matching process. Our model, namely aMV-LSTM, learns the attention based coefficients to weight words of the different input sentences, before computing their position-based representations. Experimental results, in question/answer matching and question pairs identification tasks, show that the proposed model outperforms the MV-LSTM baseline and several state-of-the-art models

High Efficiency Dye-sensitized Solar Cells Constructed with Composites of TiO2 and the Hot-bubbling Synthesized Ultra-Small SnO2 Nanocrystals

International audienceAn efficient photo-anode for the dye-sensitized solar cells (DSSCs) should have features of high loading of dye molecules, favorable band alignments and good efficiency in electron transport. Herein, the 3.4 nm-sized SnO2 nanocrystals (NCs) of high crystallinity, synthesized via the hot-bubbling method, were incorporated with the commercial TiO2 (P25) particles to fabricate the photo-anodes. The optimal percentage of the doped SnO2 NCs was found at ~7.5% (SnO2/TiO2, w/w), and the fabricated DSSC delivers a power conversion efficiency up to 6.7%, which is 1.52 times of the P25 based DSSCs. The ultra-small SnO2 NCs offer three benefits, (1) the incorporation of SnO2 NCs enlarges surface areas of the photo-anode films, and higher dye-loading amounts were achieved; (2) the high charge mobility provided by SnO2 was confirmed to accelerate the electron transport, and the photo-electron recombination was suppressed by the highly-crystallized NCs; (3) the conduction band minimum (CBM) of the SnO2 NCs was uplifted due to the quantum size effects, and this was found to alleviate the decrement in the open-circuit voltage. This work highlights great contributions of the SnO2 NCs to the improvement of the photovoltaic performances in the DSSCs

Dye-Sensitized Solar Cells Employing a Multifunctionalized Hierarchical SnO₂ Nanoflower Structure Passivated by TiO₂ Nanogranulum

We investigated a facile multifunctionalized hierarchical SnO₂ nanoflower photoelectrode passivated by a layer of TiO₂ nanogranulum. The hierarchical SnO₂ nanoflower with thin nanorod and nanosheet has a unique morphology that can afford excellent electron transport propertiesorientation overall, which results in a significant diminution in the charge diffusion route and a rapid collection in FTO substrate. The passivated photoanode not only improved the distribution of dyes in the photoelectrode and reduced the surface defects of SnO₂ photoelectrode to accommodate more dyes, but also suppressed the charge recombination and prolonged electron lifetime by introducing a barrier layer. The microstructure of the sample was investigated by X-ray diffraction (XRD), scanning electron microscopy (SEM) and transmission electron microscopy (TEM). The surface areas (<i>S</i>_BET) and pore size distribution were detected on BET measurement. The amounts of dye were calculated from UV–vis. The interfacial charge transfer process and the charge recombination were characterized by EIS and IMPS/IMVS measurements. The DSSCs assembled with multifunctionalized photoanode exhibits favorable energy conversion efficiency. The photocurrent increased from 5.44 to 12.74 mA cm², the photovoltage from 440 to 760 mV, and the fill factor from 43.58% to 57.58%. As a result, the cell’s conversion efficiency increased by a factor of 5.3 from 1.05% to 5.60%. The increase in efficiency originates from higher open-circuit potential and higher short-circuit current as well as from superior light scattering effect, long electron lifetime, and slower electron recombination